《Android Framework 之路》Android5.1 Camera Framework（二）

上一次讲解了一下CameraService的启动过程，今天梳理一下Camera预览的过程

tartPreview过程

首先，我们还是从应用层的使用入手
Camera.java (packages\apps\legacycamera\src\com\android\camera)

Thread mCameraPreviewThread = new Thread(new Runnable() {
    public void run() {
        initializeCapabilities(); //初始化参数
        startPreview(); //启动预览
    }
});

针对相机应用，采用了单独的线程来处理预览，猜测是为了加快预览显示的速度

private void startPreview() {
        ......
        // If we're previewing already, stop the preview first (this will blank
        // the screen).
        if (mCameraState != PREVIEW_STOPPED) stopPreview();

        setPreviewDisplay(mSurfaceHolder); //设置SurfaceHolder
        setDisplayOrientation();       //设置显示方向
        ......
        setCameraParameters(UPDATE_PARAM_ALL); //设置参数

        // Inform the mainthread to go on the UI initialization.
        if (mCameraPreviewThread != null) {
            synchronized (mCameraPreviewThread) {
                mCameraPreviewThread.notify();
            }
        }

        try {
            Log.v(TAG, "startPreview");
            mCameraDevice.startPreview(); //启动预览，若失败，关闭Camera
        } catch (Throwable ex) {
            closeCamera();
            throw new RuntimeException("startPreview failed", ex);
        }
        ......
    }

这里就是APP中启动预览的过程，过程必然会是
java -> JNI -> cpp，
然后通过Binder机制执行到CameraClient中的
CameraClient.cpp (av\services\camera\libcameraservice\api1)

status_t CameraClient::startPreview() {
    LOG1("startPreview (pid %d)", getCallingPid());
    return startCameraMode(CAMERA_PREVIEW_MODE);
}

这里传入的是CAMERA_PREVIEW_MODE，枚举类型是在CameraClient.h中定义的
// camera operation mode
enum camera_mode {
CAMERA_PREVIEW_MODE = 0, // frame automatically released
CAMERA_RECORDING_MODE = 1, // frame has to be explicitly released by releaseRecordingFrame()
};
第一种是针对普通的预览，第二种是针对录像

// start preview or recording
status_t CameraClient::startCameraMode(camera_mode mode) {
    LOG1("startCameraMode(%d)", mode);
    Mutex::Autolock lock(mLock);
    status_t result = checkPidAndHardware();
    if (result != NO_ERROR) return result;

    switch(mode) {
        case CAMERA_PREVIEW_MODE:
            if (mSurface == 0 && mPreviewWindow == 0) { 
                LOG1("mSurface is not set yet.");
                // still able to start preview in this case.
            }
            return startPreviewMode(); //开始预览模式
        case CAMERA_RECORDING_MODE:
            if (mSurface == 0 && mPreviewWindow == 0) {
                ALOGE("mSurface or mPreviewWindow must be set before startRecordingMode.");
                return INVALID_OPERATION;
            }
            return startRecordingMode(); //开始录像模式
        default:
            return UNKNOWN_ERROR;
    }
}

这里我们走的是预览模式

status_t CameraClient::startPreviewMode() {
    LOG1("startPreviewMode"); //LOG1,一直忘记说了，这是有log开关用过setprop可以使用
    status_t result = NO_ERROR;

    // if preview has been enabled, nothing needs to be done
    if (mHardware->previewEnabled()) { //如果已经启动预览，不必重复
        return NO_ERROR;
    }

    if (mPreviewWindow != 0) {
        //适配显示窗口的大小
        native_window_set_scaling_mode(mPreviewWindow.get(),
                NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);
        //调整帧数据的方向
        native_window_set_buffers_transform(mPreviewWindow.get(),
                mOrientation);
    }
    mHardware->setPreviewWindow(mPreviewWindow); //设置mPreviewWindow为显示窗口
    result = mHardware->startPreview();   //HAL层启动预览

    return result; //返回结果
}

这里面涉及到native_window_set_scaling_mode，native_window_set_buffers_transform，直接跟代码，看注释就可以理解，这部分涉及到显示的一些内容，这里暂时不做讲解，native_window_set_scaling_mode设置模式为NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW，native_window_set_buffers_transform是用来调整方向。
这里有一个问题，mPreviewWindow是从何而来的呢？
还记得我们在应用层的startPreview()方法中会有这么一个过程

1 2	setPreviewDisplay(mSurfaceHolder); setDisplayOrientation();

这里的setPreviewDisplay(mSurfaceHolder)，中间的过程大家可以自己跟一下，最后会执行到

status_t CameraClient::setPreviewTarget(
        const sp<IGraphicBufferProducer>& bufferProducer) {
    ......
    sp<IBinder> binder;
    sp<ANativeWindow> window;
    if (bufferProducer != 0) {
        binder = bufferProducer->asBinder();
        // Using controlledByApp flag to ensure that the buffer queue remains in
        // async mode for the old camera API, where many applications depend
        // on that behavior.
        window = new Surface(bufferProducer, /*controlledByApp*/ true); //这个家伙
    }
    return setPreviewWindow(binder, window);
}

status_t CameraClient::setPreviewWindow(const sp<IBinder>& binder,
        const sp<ANativeWindow>& window) {
    Mutex::Autolock lock(mLock);
    ......
    if (window != 0) {
        result = native_window_api_connect(window.get(), NATIVE_WINDOW_API_CAMERA);
        if (result != NO_ERROR) {
            ALOGE("native_window_api_connect failed: %s (%d)", strerror(-result),
                    result);
            return result;
        }
    }

    // If preview has been already started, register preview buffers now.
    if (mHardware->previewEnabled()) {
        if (window != 0) {
            native_window_set_scaling_mode(window.get(),
                    NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW);
            native_window_set_buffers_transform(window.get(), mOrientation);
            result = mHardware->setPreviewWindow(window);
        }
    }
    if (result == NO_ERROR) {
        // Everything has succeeded.  Disconnect the old window and remember the
        // new window.
        disconnectWindow(mPreviewWindow);
        mSurface = binder;
        mPreviewWindow = window; //这里便是赋值的操作了，后面我们操作的mPreviewWindow 
    } else {
        // Something went wrong after we connected to the new window, so
        // disconnect here.
        disconnectWindow(window);
    }
    return result;
}

是不是看的很眼熟，和startPreviewMode()的过程有点相似。这就是mPreviewWindow的赋值过程。
回到setPreviewMode()函数中，其中主要的过程是这两个
mHardware->setPreviewWindow(mPreviewWindow);
result = mHardware->startPreview();
这里都是HAL层的处理，将窗口传下去，然后启动预览，最后数据就可以投射到这个预览窗口上了。
我们继续往下看一下，
CameraHardwareInterface.h (av\services\camera\libcameraservice\device1)

/** Set the ANativeWindow to which preview frames are sent */
status_t setPreviewWindow(const sp<ANativeWindow>& buf)
{
    ALOGV("%s(%s) buf %p", __FUNCTION__, mName.string(), buf.get());

    if (mDevice->ops->set_preview_window) {
        mPreviewWindow = buf;
        mHalPreviewWindow.user = this;
        ALOGV("%s &mHalPreviewWindow %p mHalPreviewWindow.user %p", __FUNCTION__,
                &mHalPreviewWindow, mHalPreviewWindow.user);
        return mDevice->ops->set_preview_window(mDevice,
                buf.get() ? &mHalPreviewWindow.nw : 0);
    }
    return INVALID_OPERATION;
}

/**
  * Start preview mode.
  */
 status_t startPreview()
 {
     ALOGV("%s(%s)", __FUNCTION__, mName.string());
     if (mDevice->ops->start_preview)
         return mDevice->ops->start_preview(mDevice);
     return INVALID_OPERATION;
 }

这是一个空壳，我们去看具体的实现，这里我们看下android5.1源码中Qcom的实现，由于针对HAL层的不同厂商有不同的处理方式，在这里我们就随便找个目录下的进行分析，旨在看流程，理解一些基础的内容，
QCamera2Hal.cpp （\device\asus\flo\camera\qcamera2\hal）

#include "QCamera2Factory.h"

static hw_module_t camera_common = {
    tag: HARDWARE_MODULE_TAG,
    module_api_version: CAMERA_MODULE_API_VERSION_1_0,
    hal_api_version: HARDWARE_HAL_API_VERSION,
    id: CAMERA_HARDWARE_MODULE_ID,
    name: "QCamera Module",
    author: "Qualcomm Innovation Center Inc",
    methods: &qcamera::QCamera2Factory::mModuleMethods,
    dso: NULL,
    reserved:  {0},
};

camera_module_t HAL_MODULE_INFO_SYM = {
    common: camera_common,
    get_number_of_cameras: qcamera::QCamera2Factory::get_number_of_cameras,
    get_camera_info: qcamera::QCamera2Factory::get_camera_info,
    set_callbacks: NULL,
    get_vendor_tag_ops: NULL,
    open_legacy: NULL,
    reserved: {0}
};

这里提一下HAL_MODULE_INFO_SYM这个东西，本身就是一个定义在hardware.h下的一个宏，看注释，意思很明显

1 2	define HAL_MODULE_INFO_SYM HMI //.so中将一个符号HMI,获取此符号的地址，就获取到了对应的hw_module_t地址

HAL_MODULE_INFO_SYM，这个是HAL 编译生成的so的入口，CameraService会获取这个来操作so
camera_common是针对HAL规范定义的一些内容。
Camera的open指向的是&qcamera::QCamera2Factory::mModuleMethods中的open方法，如下

1
2
3

struct hw_module_methods_t QCamera2Factory::mModuleMethods = {
    open: QCamera2Factory::camera_device_open,
};

这个方法中打开设备节点，我们可以看到HAL层中open的过程是很有讲究的，也不能这么说，应为HAL的处理方式基本上都是如此。

int QCamera2Factory::camera_device_open(
    const struct hw_module_t *module, const char *id,
    struct hw_device_t **hw_device)
{
    if (module != &HAL_MODULE_INFO_SYM.common) {
        ALOGE("Invalid module. Trying to open %p, expect %p",
            module, &HAL_MODULE_INFO_SYM.common);
        return INVALID_OPERATION;
    }
    if (!id) {
        ALOGE("Invalid camera id");
        return BAD_VALUE;
    }
    return gQCamera2Factory.cameraDeviceOpen(atoi(id), hw_device);
}

int QCamera2Factory::cameraDeviceOpen(int camera_id,
                    struct hw_device_t **hw_device)
{
    int rc = NO_ERROR;
    if (camera_id < 0 || camera_id >= mNumOfCameras)
        return BAD_VALUE;
    //到这里才是真正的HAL层的创建，可见HAL层的创建和open操作是相关的
    QCamera2HardwareInterface *hw = new QCamera2HardwareInterface(camera_id);
    if (!hw) {
        ALOGE("Allocation of hardware interface failed");
        return NO_MEMORY;
    }
    rc = hw->openCamera(hw_device);
    if (rc != NO_ERROR) {
        delete hw;
    }
    return rc;
}

之前在CameraService过程中提到的CameraHardwareInterface空壳就是为这个QCamera2HardwareInterface准备的，具体实现全部都在这个类中。
关于QCamera2HardwareInterface的内容我们在后面会讲到，这里暂且先放一下，接着上面的
mHardware->setPreviewWindow(mPreviewWindow);
result = mHardware->startPreview();
经过CameraHardwareInterface后
mDevice->ops->set_preview_window(mDevice, buf.get() ? &mHalPreviewWindow.nw : 0);
mDevice->ops->start_preview(mDevice);
然后经过QCamera2HardwareInterface中的mCameraOps函数指针对应表
set_preview_window: QCamera2HardwareInterface::set_preview_window
start_preview: QCamera2HardwareInterface::start_preview
所以会调用到

int QCamera2HardwareInterface::set_preview_window(struct camera_device *device,
        struct preview_stream_ops *window)
{
    int rc = NO_ERROR;
    QCamera2HardwareInterface *hw =
        reinterpret_cast<QCamera2HardwareInterface *>(device->priv);
    if (!hw) {
        ALOGE("%s: NULL camera device", __func__);
        return BAD_VALUE;
    }

    hw->lockAPI();
    rc = hw->processAPI(QCAMERA_SM_EVT_SET_PREVIEW_WINDOW, (void *)window);
    if (rc == NO_ERROR) {
        hw->waitAPIResult(QCAMERA_SM_EVT_SET_PREVIEW_WINDOW);
        rc = hw->m_apiResult.status;
    }
    hw->unlockAPI();

    return rc;
}

这里会经过一轮状态机，暂时先不讲，然后执行到
int QCamera2HardwareInterface::setPreviewWindow(
struct preview_stream_ops window)
{
mPreviewWindow = window;
return NO_ERROR;
}
同理，startPreview也会执行到
int QCamera2HardwareInterface::startPreview()
{
int32_t rc = NO_ERROR;
ALOGD(“%s: E”, *func);
// start preview stream
if (mParameters.isZSLMode() && mParameters.getRecordingHintValue() !=true) {
rc = startChannel(QCAMERA_CH_TYPE_ZSL);
} else {
rc = startChannel(QCAMERA_CH_TYPE_PREVIEW);
}
ALOGD(“%s: X”, **func);
return rc;
}
这里开启通道，可以理解成数据通道，ZSL是之前没有的，所谓ZSL就是触发拍照后不停止预览。
这里看到会根据当前是都支持ZSL模式而进入不同的通道，我们这里就看QCAMERA_CH_TYPE_PREVIEW，startChannel

int32_t QCamera2HardwareInterface::startChannel(qcamera_ch_type_enum_t ch_type)
{
    int32_t rc = UNKNOWN_ERROR;
    if (m_channels[ch_type] != NULL) {
        rc = m_channels[ch_type]->start();
    }

    return rc;
}

m_channels是不同的通道的实例的数组，这里如果没有PREVIEW的channel就直接return，岂不是无法启动预览，这个流程感觉有点不对劲。但是这整个过程跟下来也没有看到m_channels相关的初始化过程。

这个问题出在我刚才从CameraHardwareInterface跟到QCamera2HardwareInterface的时候跳过的一个内容—–状态机，在状态机中会执行一个preparePreview()的操作

int32_t QCamera2HardwareInterface::preparePreview()
{
    int32_t rc = NO_ERROR;

    if (mParameters.isZSLMode() && mParameters.getRecordingHintValue() !=true) {
        rc = addChannel(QCAMERA_CH_TYPE_ZSL); //这里我们就添加了一个channel,当然这里是ZSL的
        if (rc != NO_ERROR) {
            return rc;
        }
    } else {
        bool recordingHint = mParameters.getRecordingHintValue(); //recording
        if(recordingHint) {
            rc = addChannel(QCAMERA_CH_TYPE_SNAPSHOT);  //录像中是可以拍照的，需要snapshot channel
            if (rc != NO_ERROR) {
                return rc;
            }

            rc = addChannel(QCAMERA_CH_TYPE_VIDEO); //video channel
            if (rc != NO_ERROR) {
                delChannel(QCAMERA_CH_TYPE_SNAPSHOT);
                return rc;
            }
        }

        rc = addChannel(QCAMERA_CH_TYPE_PREVIEW); //添加preview  channel
        if (rc != NO_ERROR) {
            if (recordingHint) {
                delChannel(QCAMERA_CH_TYPE_SNAPSHOT);
                delChannel(QCAMERA_CH_TYPE_VIDEO);
            }
            return rc;
        }

    }

    return rc;
}

在addchannel()的过程中会根据不同的channel类型创建不同的实例，这里我们直接看从addChannel()转到的addPreviewChannel()函数

int32_t QCamera2HardwareInterface::addPreviewChannel()
{
    int32_t rc = NO_ERROR;
    QCameraChannel *pChannel = NULL; //初始化一个QCameraChanel，后面要使用
    if (m_channels[QCAMERA_CH_TYPE_PREVIEW] != NULL) {
        // if we had preview channel before, delete it first
        delete m_channels[QCAMERA_CH_TYPE_PREVIEW]; //如果之前preview channel存在，干掉
        m_channels[QCAMERA_CH_TYPE_PREVIEW] = NULL;
    }
    pChannel = new QCameraChannel(mCameraHandle->camera_handle,
                                  mCameraHandle->ops); 
    //new 一个新的channel
    .....
    // meta data stream always coexists with preview if applicable
    rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_METADATA,
                            metadata_stream_cb_routine, this);
    //添加metadata stream cb
    if (rc != NO_ERROR) {
        ALOGE("%s: add metadata stream failed, ret = %d", __func__, rc);
        delete pChannel;
        return rc;
    }

    if (isNoDisplayMode()) { //判断是否为不需要显示的模式
        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,
                                nodisplay_preview_stream_cb_routine, this);
    } else {
        //这里添加preview stream cb到channel中
        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,
                                preview_stream_cb_routine, this);
    }
    if (rc != NO_ERROR) {
        ALOGE("%s: add preview stream failed, ret = %d", __func__, rc);
        delete pChannel;
        return rc;
    }

    m_channels[QCAMERA_CH_TYPE_PREVIEW] = pChannel; //维护m_channels数据
    return rc;
}

这里注册的preview_stream_cb_routine回调，这之后的过程我们暂时先不去看，了解这部分之后，回到之前chanel 的start()，最后会执行到QCameraChannel::start()方法，这里往下的内容我们暂时不往下看，知道这个过程中会执行数据采集，然后返回给HAL层就行了，HAL针对底层返回的数据，我们在哪里获取，做什么对应的处理呢？找到之前注册的Callback.
QCamera2HWICallbacks.cpp (\device\asus\flo\camera\qcamera2\hal)

void QCamera2HardwareInterface::preview_stream_cb_routine(mm_camera_super_buf_t *super_frame,
                                                          QCameraStream * stream,
                                                          void *userdata)
{
    ALOGD("[KPI Perf] %s : BEGIN", __func__);
    int err = NO_ERROR;
    QCamera2HardwareInterface *pme = (QCamera2HardwareInterface *)userdata;
    QCameraGrallocMemory *memory = (QCameraGrallocMemory *)super_frame->bufs[0]->mem_info;
    ......
    mm_camera_buf_def_t *frame = super_frame->bufs[0];
    ......
    if (!pme->needProcessPreviewFrame()) {
        ALOGE("%s: preview is not running, no need to process", __func__);
        stream->bufDone(frame->buf_idx);
        free(super_frame);
        return;
    }
    if (pme->needDebugFps()) {
        pme->debugShowPreviewFPS();
    }
    int idx = frame->buf_idx;
    pme->dumpFrameToFile(frame->buffer, frame->frame_len,
                         frame->frame_idx, QCAMERA_DUMP_FRM_PREVIEW);
    //这里的注释很明显，displayer buffer而这个buffer就是我们需要投射到屏幕上的数据
    // Display the buffer.
    int dequeuedIdx = memory->displayBuffer(idx); //这部分涉及到显示的过程，这里不做赘述
    if (dequeuedIdx < 0 || dequeuedIdx >= memory->getCnt()) {
        ALOGD("%s: Invalid dequeued buffer index %d from display",
              __func__, dequeuedIdx);
    } else {
        // Return dequeued buffer back to driver
        err = stream->bufDone(dequeuedIdx);
        if ( err < 0) {
            ALOGE("stream bufDone failed %d", err);
        }
    }
    //针对上层设置的datacallback过程做些处理
    // Handle preview data callback
    if (pme->mDataCb != NULL && pme->msgTypeEnabledWithLock(CAMERA_MSG_PREVIEW_FRAME) > 0) {
        ......
        qcamera_callback_argm_t cbArg;
        memset(&cbArg, 0, sizeof(qcamera_callback_argm_t));
        cbArg.cb_type = QCAMERA_DATA_CALLBACK;
        cbArg.msg_type = CAMERA_MSG_PREVIEW_FRAME;
        cbArg.data = data;
        if ( previewMem ) {
            cbArg.user_data = previewMem;
            cbArg.release_cb = releaseCameraMemory;
        }
        cbArg.cookie = pme;
        pme->m_cbNotifier.notifyCallback(cbArg); //封装完之后往上甩
    }

    free(super_frame);
    ALOGD("[KPI Perf] %s : END", __func__);
    return;
}